Storage system

ABSTRACT

In a storage system having a plurality of virtualization apparatuses that allocate a storage area which a storage device has, form a plurality of virtual volumes, and process input-output from a host processor to one of the virtual volumes, a request for completing all the input-output being processed that is received from the host processor and temporarily holding the input-output processing received subsequently is issued to the plurality of virtualization apparatuses, when a completion report of the processing of the input-output in response to the request is received from the plurality of virtualization apparatuses and the completion report is received from all the virtualization apparatuses to which the request was issued, an instruction of an allocation change of the storage area of the storage device is sent to all the virtualization apparatuses, and when the completion report of the allocation change is received from all the virtualization apparatuses, the storage area of the storage device is allocated to the virtual volume by sending an instruction for releasing a state of the input-output held temporarily to all the virtualization apparatuses.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application is a Continuation Application of U.S.application Ser. No. 10/735,155, filed Dec. 12, 2003, which is relatedto and claims priority from Japanese Patent Application No. 2003-346608,filed Oct. 6, 2003, both of which are incorporated by reference hereinin their entirety for all purposes.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a storage system, and more particularlyto a change in the allocation of a real storage area to a virtual volumein a storage system having a virtualization apparatus of a redundantconfiguration.

2. Description of the Prior Arts

In the so-called SAN (Storage Area Network), a system that sets aplurality of virtual volumes with reference to the storage area of astorage device and uses their volumes from a host processor via anetwork is known.

Regarding a storage virtualization apparatus that virtualizes thestorage device connected through the network and enables input-output(I/O) from the host processor, for example, there is such a system asdisclosed in Japanese Unexamined Patent Application Publication No. Hei2000-242434 (Patent Reference 1). According to this art, a storagedevice to which a virtual storage area provided to a host is allocatedis changed by installing switching equipment 20 between a storage devicesystem 1 and a host 30 and the switching equipment 20 changes thevirtualization setting of a virtual storage device system provided tothe host 30, thereby to change the storage device to which a virtualstorage area to be provided to a host is allocated.

The configuration information about the storage virtualization apparatus(for example, array disk switch) group described in Patent Reference 1is managed independently for each storage virtualization apparatus.

In such a storage virtualization system, the modification of theconfiguration information during system operation changes thedestination during input-output processing and causes data corruptionand an input-output fault. Accordingly, a method for reducing thestorage virtualization apparatuses that operate concurrently during aconfiguration change to only one apparatus can be considered, but aproblem that concentration of a load or fault tolerance decreases isarisen. The aforementioned Patent Reference 1 does not refer to themodification of configuration information indicating that one volumeshifts to another volume while the storage device system is operating.

BRIEF SUMMARY OF THE INVENTION

An object of the present invention is to prevent data corruption and aninput-output fault during system operation and change the configurationinformation about a storage virtualization apparatus in a storage systemhaving a virtualization apparatus of redundant configuration.

The present invention is constructed in a plurality of virtualizationapparatuses that process input-output to/from a host processor to avirtual volume on the condition that a request for temporarily holdingthe input-output processing accepted from the host processor after acertain point of time is issued to the plurality of virtualizationapparatuses and a report indicating that the ongoing input-outputprocessing was completed in regard to this request was received fromeach virtualization apparatus. On the condition, the present inventionreleases an input-output state held temporarily after having changed theallocation of the storage area of a storage device to eachvirtualization apparatus and having accepted a completion report of theallocation change from each virtualization apparatus.

As a desirable example concerning a storage system, the storage systemhaving a storage device that can specify a plurality of storage areasand a plurality of virtualization apparatuses that allocate a storagearea which this storage device has, form a plurality of virtual volumes,process the input-output from a host processor to one of the virtualvolumes, and includes a configuration change controller for changing anallocation configuration of storage area of the storage device to thevirtual volume. The configuration change controller has a means forrequesting a temporary hold of the input-output to all thevirtualization apparatuses before a configuration change and a means forallowing all the virtualization apparatuses that received this requestto complete the input-output being processed and to subsequently shiftto a state of temporarily holding an input-output request from the hostprocessor subsequently, then to return a completion report to theconfiguration change controller. The configuration change controllerhas, when receiving the completion report from the previous pluralvirtualization apparatuses to which a request was issued, a means forinstructing an allocation change of the storage area of the storagedevice to the virtual volume to the virtualization apparatus.

Further, as a desirable example regarding a plurality of virtualizationapparatuses, they have a configuration change control program forchanging a configuration of associating a virtual volume with a storagearea that becomes a real area of the storage device and a firstprocessor that executes the configuration change control program. Thisconfiguration change control program has, before changing theconfiguration of associating the virtual volume with the storage areathat becomes the real area of the storage device, a means for requestingan input-output temporary hold to another virtualization apparatus. Theother virtualization apparatus that received the request has a means forcompleting the input-output being processed and subsequently shifting toa state of temporarily holding an input-output request from a hostprocessor, and returning a completion report. The configuration changecontrol program has, when receiving the completion report from the othervirtualization apparatus, a means for instructing an allocation changeof the storage area of the storage device to the virtual volume to theother virtualization apparatuses, a means for receiving the completionreport of the allocation change from another virtualization apparatus,and a means for sending an instruction for releasing the state of theinput-output held temporarily to the other virtualization apparatus.

Furthermore, as an example of the configuration concerning a storagedevice, the storage device has a plurality of storage areas forproviding a real storage area and a virtualization apparatus thatallocates the plurality of storage areas, forms a plurality of virtualvolumes, and processes the input-output from a host processor to one ofthe virtual volumes. This virtualization apparatus has, before changinga configuration of associating the virtual volume with the storage areathat becomes a real area of the storage device, a means for requestingan input-output temporarily hold to another virtualization apparatus.The other virtualization apparatus that received the request has a meansfor completing the input-output being processed and subsequentlyshifting to a state of temporarily holding an input-output request fromthe host processor, and returning a completion report. Thevirtualization apparatus has, when receiving the completion report fromthe other virtualization apparatus, a means for instructing anallocation change of the storage area in regard to the virtual volume tothe other virtualization apparatus, a means for receiving the completionreport of the allocation change from the other virtualization apparatus,and a means for sending an instruction for releasing the state of theinput-output held temporarily to the other virtualization apparatus.

As a more desirable example, data can migrate from one storage device toanother storage device during system operation by copying the databetween the storage devices synchronizing with the change ofconfiguration information. Moreover, even a virtualization apparatus nothaving a copy function can migrate data by allowing the storage deviceto implement a configuration change control function and theaforementioned copy processing function.

According to the present invention, the configuration information of avirtual volume can be changed during system operation by preventing asmuch influence of an input-output temporary hold as possible.Consequently, data can migrate from one storage device to anotherstorage device during system operation. Moreover, even a virtualizationapparatus not having the copy function can shift data. A storage devicethat can change an allocation destination of the virtual volume can berealized during operation in a redundant configuration.

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred embodiments of the present invention will be described indetail based on the followings, wherein:

FIG. 1 is a drawing showing the overall configuration of a storagesystem according to a first embodiment;

FIG. 2 is a drawing showing the internal configuration of avirtualization switch 11 of FIG. 1;

FIG. 3 is a drawing showing the internal configuration of aconfiguration change controller 16 of FIG. 1;

FIG. 4 is a drawing showing the table of a configuration information 221in the storage system;

FIG. 5 is a drawing showing the communication protocol between aconfiguration change controller 16 and a virtualization switch 11 in thestorage system of FIG. 1;

FIG. 6 is a flowchart showing the processing of a configuration changecontrol program 212 in a configuration change controller 16;

FIG. 7 is a flowchart showing the processing of a configurationmanagement program 211 in a virtualization switch 11;

FIG. 8 is a flowchart showing processing of an I/O processing program213 in the virtualization switch 11;

FIG. 9 is a drawing showing the internal configuration of thevirtualization switch 11 according to a second embodiment;

FIG. 10 is a flowchart showing the processing of a configuration changecontrol program 212 in the second embodiment;

FIG. 11 is a flowchart showing the processing operation of arbitrationprocessing 600 in FIG. 10;

FIG. 12 is a drawing showing the internal processing of thevirtualization switch 11 according to a third embodiment;

FIG. 13 is a drawing showing the communication protocol between theconfiguration change controller 16 and the virtualization switch 11 inthe third embodiment;

FIG. 14 is a drawing showing an example of a temporary hold controltable 223 in FIG. 12;

FIG. 15 is a drawing showing the table of the configuration information221 in FIG. 12;

FIG. 16 is a flowchart showing the processing of the configurationmanagement program 212 in FIG. 12;

FIG. 17 is a flowchart showing the processing of the configurationmanagement program 211 in FIG. 12;

FIG. 18 is a flowchart showing the processing of the I/O processingprogram 213 in FIG. 12;

FIG. 19 is a drawing showing the internal processing of thevirtualization switch 11 in a fourth embodiment;

FIG. 20 is a drawing showing the table of the configuration information221 in FIG. 19;

FIG. 21 is a flowchart showing the processing of the configurationchange control program 212 in FIG. 19;

FIG. 22 is a flowchart showing the details of copy processing 631 in theflowchart of FIG. 21;

FIG. 23 is a flowchart showing the processing of the configurationmanagement program 211 in FIG. 19;

FIG. 24 is a flowchart showing the processing of the I/O processingprogram 213 in FIG. 19;

FIG. 25 is a flowchart showing the processing of a copy processingprogram 214 in FIG. 19;

FIG. 26 is a drawing for describing an operation principle of datamigration in the fourth embodiment;

FIG. 27 is a drawing showing the overall configuration of the storagesystem in a fifth embodiment;

FIG. 28 is a flowchart showing the processing of the configurationchange control program 212 in the fifth embodiment; and

FIG. 29 is a drawing showing the overall configuration of the storagesystem in a sixth embodiment.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the present invention will be described below withreference to the drawings.

First, a first embodiment is described with reference to FIGS. 1 to 8.

FIG. 1 is a drawing showing the overall configuration of a storagesystem.

The storage system connects at least one host processor 12, a pluralityof storage devices 13, a plurality of virtualization switches 11, aconfiguration change controller 16, and a management console 14 to anetwork 15 such as a LAN.

The host processor 12 is a computer that uses data stored in the storagedevice 13. The host processor 12 may be a file server having a functionprovided to another computer that does not connect a storage area whichthe virtualization switch 11 provides to the virtualization switch 11.

The storage device 13 is provided with a memory unit 13 a or a memoryunit system 13 b. In this case, the memory unit 13 a is a single memoryunit such as a hard disk drive or a DVD drive. The memory unit system 13b is a storage subsystem having the plural memory units 13 a and acontroller 1301 that controls these memory units. A memory unit 131constructs the storage area of the memory unit 13 a as a logical unit(hereafter referred to as an “LU”) 131. The LU 131 is a logical storagearea, and the device connected to the storage device 13 such as the hostprocessor 12, is recognized as one logically independent storage device.

Further, the logical unit 131 is provided with a plurality of partlylogical storage areas (hereafter referred to as “real areas”) 132. Eachof the real areas 132 corresponds to a physical storage area which thestorage device 13 has. The size of the real area 132 is arbitrary andthe range is an area having a continuous address.

The virtualization switch 11 is connected to another device through acommunication line or by a switch as shown in the drawing and cancommunicate with another device. Further, the virtualization switch 11is a virtualization apparatus that collects (“virtualizes”) the storageareas which the plural storage devices 13 connected to thevirtualization switch 11 itself has as one or more storage areas. Thenthe virtualization switch 11 provides a virtualized storage area to thehost processor 12 connected to the virtualization switch 11. The virtualstorage area which the virtualization switch 11 provides to the hostprocessor 12 is hereafter referred to as a virtual volume 100.

A protocol such as a fibre channel, is used in the communication line orswitch used between the virtualization switch 11 and the host processor12 and between the virtualization switch 11 and the storage device 13.In this case, the communication line or switch used may be thecommunication line or protocol used in a local area network. Thevirtualization switch 11 is connected between the host processor 12 andthe storage device 13 and has a function of transferring a command thehost processor 12 issues to the side of the storage device 13.

The virtual volume 100 is a virtualized storage area having at least theone real area 132. The virtualization switch 11 can provide at least onevirtual volume 100 to the host processor 12. A unique identifier(hereafter referred to as a “virtual volume identifier”) is assigned toeach virtual volume 100 in the virtualization switch 11 for specifying avirtual volume. Moreover, a continuous address is assigned to thestorage area of each virtual volume 100. The host processor 12 specifiesan address indicating a virtual volume identifier and a location in thevirtual volume 100 and accesses data stored in the storage device 13instead of directly specifying the real area 132 within the LU 131 ofthe storage device 13.

The management console 14 is such a personal computer (PC) that is usedby a system administrator to create the virtual volume 100 and has adisplay device or an input device, and a memory. The management console14 is connected to the virtualization switch 11 via the LAN 15 and cancommunicate with each other.

The configuration change controller 16 controls the virtualizationswitch 11 and controls configuration information, that is, the change ofassociating the virtual volume 100 with the real area 132. Theconfiguration change controller 16 can have a PC or a server, forexample, and is connected to the virtualization switch 11 via the LAN15, which can communicate with each other.

FIG. 2 shows the internal configuration of the virtualization switch 11of FIG. 1.

The virtualization switch 11 is provided with an input port 240, anoutput port 250, a transfer unit 230, a processor 210, a memory 220, abus 270, and a communication unit 260. The transfer unit 230, processor210, memory 220, and communication unit 260 are all connected to the bus270, and send and receive data to/from each other.

The input port 240 connects a communication line through which thevirtualization switch 11 communicates with the host processor 12. Theoutput port 250 connects a communication line through which thevirtualization switch 11 communicates with the storage device 13.Further, the element constructing the input port 240 and the output port250 may be located in the same hardware. In this case, the user selectswhich port is used as the input port or the output port.

The transfer unit 230 has an internal memory and holds a transferinformation table 231 in the memory. The transfer information table 231stores the interrelationship between the host processor 12 that cancommunicate with the virtualization switch 11 via each input port 240and the storage device 13 that can communicate with the virtualizationswitch 11 via each output port 250.

The transfer unit 230 refers to the transfer information table 231 andtransfers an input-output request received from the host processor 12via the input port 240 to the output port 250 that is used for thecommunication between the storage device 13 and the virtualizationswitch 11 of a requesting destination. Further, the transfer unit 230transports the response information or data received from the storagedevice 13 via the output port 250 to the input port 240 that is used forthe communication between the host processor 12 and the virtualizationswitch 11 which ought to receive the information or data. When theinput-output request received from the host processor 12 is aninput-output request to the virtual volume 100, the transfer unit 230enqueues the input-output request to an input queue 241 to be describedlater and requests the processing from the processor 210. Further, thetransfer unit transfers the input-output request stacked on an outputqueue 251 to be described later to the storage device 13 via the outputport.

The processor 210 executes a program stored on the memory 220 andperforms the input-output processing or the change processing of theconfiguration information for the virtual volume 100 from the hostprocessor 12.

The memory 220 stores the program the processor 210 executes and theinformation necessary for the execution. The program and data the memory220 stores includes a configuration management program 211, an I/Oprocessing program 213, the input queue 241, the output queue 251, aon-hold queue 242, a in-process queue 252, configuration information221, and a configuration information difference 222.

The configuration management program 211 receives a request from theconfiguration change controller 16 and performs input-output temporaryhold and restart processing or configuration information changeprocessing.

The I/O processing program 213 processes the input-output processing forthe virtual volume 100 from the host processor 12, that is, converts theinput-output to the input-output for the storage device 13 and transfersit.

The input queue 241 allows the transfer unit 230 to stack theinput-output request for the virtual volume 100. The output queue 251stacks the input-output request for the storage device the I/Oprocessing program 213 processed. The number of input queues 241 and thenumber of output queues 251 are optional.

The on-hold queue 242 stores the input-output request for the virtualvolume 100 accepted when setting the virtualization switch 11 in a state(I/O temporary hold state) at which the input-output processing is heldtemporarily. The in-process queue 252 stores the input-output requestfor the virtual volume 100 processed by the virtualization switch 11 andtransferred to the storage device 13 until the input-output iscompleted.

The configuration information 221 is the table for associating thevirtual volume 100 with the real area 132. The configuration informationdifference 222 is a buffer that records the difference before and afterthe change when the configuration information 221 is changed.

In this embodiment, one each of the on-hold queue 242, in-process queue252, configuration information 221, and configuration informationdifference 222 is provided for every virtualization switch 11.

The communication unit 260 enables the processor 210 to communicate withthe configuration change controller 16 and the management console 14 viathe LAN 15.

FIG. 3 shows the internal configuration of the configuration changecontroller 16 of FIG. 1.

The configuration change controller 16 is a management server, forexample, and is provided with a processor 161, a memory 162, the bus270, and the communication unit 260. The processor 161, memory 162, andcommunication unit 260 are all connected to the bus 270, and send andreceive data one after another.

The processor 161 executes the program stored on the memory 162 andcontrols the configuration change of the virtualization switch 11.

The memory 162 stores the program the processor 161 executes or theinformation necessary for the execution. The program and data the memory162 stores includes the configuration change control program 212,configuration information 221, and configuration information difference222.

The configuration change control program 212 controls the configurationchange of the virtualization switch 11.

FIG. 4 shows the table of the configuration information 221 in a storagesystem. The configuration information table 221 is provided in thevirtualization switch 11 and the configuration change controller 16.

The configuration information 221 is provided with entries that includea virtual volume address 41, an offset 42, a size 43, an LU address 44,and an offset 45. Each entry is associated with a real area 132 and apartial area on the virtual volume 100 to which the real area 132 isallocated. The LU address 44 indicates the information for allowing thevirtualization switch 11 to identify the LU 131 including the real area132 that corresponds to the entry. The offset 45 indicates the startaddress on the LU 131 of the real area 132, and the size 43 indicatesthe size of the real area 132.

The virtual volume address 41 indicates the information for allowing thehost processor 12 to identify the virtual volume 100, and the offset 42indicates the start address on the virtual volume 100 of the partialarea that corresponds to the entry. The virtual volume address 41 andthe LU address 44 specifically use a pair of a WWN (World Wide Name) orport ID, and a LUN (Logical Unit Number) of a fibre channel.

Next, the configuration change in this storage system is describedbriefly with reference to FIG. 5. FIG. 5 shows the communicationprotocol between the configuration change controller 16 and thevirtualization switch 11 in the storage system.

When an instruction of a configuration information change request isinput from the management console 14, the instruction is transferred tothe configuration change controller 16 (501). Then the processing ofsteps 502 to 505 is executed and the input-output processing issued bythe host processor 12 is held temporarily. This is because a conflictoccurs if the input-output being processed exists before and after aconfiguration change. If the configuration information is changed duringinput-output processing in this manner, the input-output cannot beassociated with the correct real area 132 and it may cause a datacorruption. This data corruption is prevented from the followingprocessing.

That is, the configuration change controller 16 that received aconfiguration instruction issues a request that temporarily holds theinput-output processing (I/O temporary hold request) to allvirtualization switches 11 (502). The virtualization switch 11 thatreceived this hold request holds the input-output being processed andsubsequent input-output processing temporarily and waits for theexecution completion of the input-output being processed (503). Afterthe execution is completed, the completion of the processing of the step503 (I/O temporary hold completion report) is reported to theconfiguration change controller 16 (504). Subsequently, theconfiguration change controller 16 waits for the completion report fromall the virtualization switches 11 (505).

The aforementioned processing enabled the change of configurationinformation. Subsequently, the processing of steps 506 to 509 isexecuted and the configuration information of all the virtualizationswitches 11 is changed collectively. This is because of the possibilityof data corruption occurring when the input-output that corresponds tothe same virtual volume is associated with the different real area 132by the virtualization switch 11 in charge of processing if theconsistency of the configuration information which the virtualizationswitch 11 has is not obtained.

The configuration change controller 16 sends a configuration informationdifference and a configuration information change request that used thedifference to all the virtualization switches 11 (506). Thevirtualization switch 11 that received the request changes theconfiguration information (507). Then after the configuration change wascompleted, the virtualization switch reports the completion of theconfiguration information change to the configuration change controller16 (508). Subsequently, the configuration change controller 16 waits forthe completion report from all the virtualization switches 11 (509).

Finally, the processing of steps 510 to 513 is executed and theinput-outputs held temporarily in the step 503 begin being restarted.

The configuration change controller 16 issues a request (I/O restartrequest) that restarts the input-output processing held temporarily toall the virtualization switches 11 (510). The virtualization switch 11that received the request restarts the input-outputs that are heldtemporarily (511). Then the virtualization switch reports the completionof the restart processing to the configuration change controller 16(512). Subsequently, the configuration change controller 16 waits forthe completion report from all the virtualization switches 11 (513) andreports the completion to a management console (514).

The configuration change is enabled without generating a conflict of theconfiguration information during input-output processing (that is,system operation) or a conflict of the configuration information betweenthe plural virtualization switches 11 in this manner.

Next, the processing operation of the configuration change controlprogram 212 in the configuration change controller 16 is described withreference to FIG. 6.

First, the processor 161 issues an I/O temporary hold request to all thevirtualization switches 11 and waits for a completion report from allthe virtualization switches 11 (601). When queuing was completednormally (602 “Y”), the change of the configuration information 221 isenabled. In this case, the processor 161 sends a configuration changerequest and the configuration information difference 222 to all thevirtualization switches 11 and changes the configuration information221. Then the processor waits for the completion report from all thevirtualization switches 11 (603). When the queuing was completednormally (604 “Y”), the configuration information 221 is changednormally. Accordingly, to restart the input-output being held, theprocessor 161 sends an I/O restart request to all the virtualizationswitches 11 and waits for the completion report from all thevirtualization switches 11 (605). When the queuing is completed normally(606 “Y”), the processor determines that all change processing hassucceeded and returns a message indicating that the processing succeededto the management console 14 (607). The aforementioned is a flow of thenormal processing of the configuration change control program 212.

On the other hand, when the virtualization switch 11 did not complete anI/O temporary hold request (602 “N”), it determines whether subsequentprocessing continues or not (608).

In this embodiment, when there are two or more of virtualizationswitches 11 to which a response of success was returned, the processingcontinues (608 “Y”). To prevent the virtualization switch 11 to whichthe response of success was not returned from being used, thevirtualization switch is removed from a control range (611) and theprocessing continues from the step 603. When several virtualizationswitches 11 are removed from the control range in the step 611, they areutilized based on the incorrect configuration information 221 if theinput-output of these virtualization switches 11 is restartedincorrectly. Accordingly, the processor 161 reports the virtualizationswitch 11 that was removed from the control range to the managementconsole 14 in the step 607 and displays a message indicating “Severalvirtualization switches were stopped because the synchronization of theinput-output temporary hold failed and configuration could not bechanged during a configuration change” on the display device of themanagement console 14”.

Further, when the processing does not continue (608 “N”), the processor161 first sends an I/O restart request to all the virtualizationswitches 11 and waits for the completion report from all thevirtualization switches 11 (609), then determines whether the processingof the step 601 and later steps is retried or not (610). In this case,when the error in the step 602 was a timeout, this timeout is consideredto result from the fact that because the virtualization switch 11 isprocessing a large amount of input-output, their completion can hardlybe waited for. Accordingly, because there is the possibility of thiscompletion wait proving successful by a retry, the retry is performedonly for a predetermined count in this embodiment if there are no othererrors (610 “Y”). In the step 610 and later steps, the processor 161repeats processing from the step 601.

On the other hand, when the processing is not retried (610 “N”), ifthere is the virtualization switch 11 to which a response of success wasnot returned in the step 609, this switch is removed from the controlrange (617). Then the occurrence of an error and its cause are reportedto the management console 14 and the management console displays amessage indicating “An input-output completion wait of a virtualizationswitch failed” (616). More desirably, the management console 14 displaysa message prompting the administrator to “Retry a configuration changewhen the input-output frequency from the host processor 12 to thevirtual volume 100 is low”.

Further, when error detection, that is, the change of the configurationinformation 221 failed in the step 604, the processor 161 returns to theoriginal configuration information 221 (612, 613) and restarts theinput-output (614, 615). Then the processor reports the occurrence of anerror and its cause to the management console 14, and, for example,displays a message indicating “The administrator failed in the change tothe specified configuration information” (616).

When an error was detected in the processing of the steps 613 and 615(for “N”), subsequent processing cannot continue. Accordingly, theprocessor 161 removes the virtualization switch 11 to which the responseof success was not returned from the control range (617) and performsthe processing of the step 616. For example, the processor displays amessage indicating “The administrator failed in the change to theconfiguration information and failed in even recovery processing”. Theprocessor 161 controls the change of the configuration information 221as described above.

Next, the processing operation of the configuration management program211 in the virtualization switch 11 is described with reference to FIG.7.

The configuration management program 211 is called when thevirtualization switch 11 received any of an I/O temporary hold request,an I/O restart request, and a configuration information change requestfrom the configuration change controller 16. First, when the processor210 received the I/O temporary hold request (701 “Y”), the processorcalls the I/O processing program 213 and sets the I/O in a temporaryhold state (702). Then when the transition to the temporary hold statesucceeded (703 “Y”), the processor 210 returns a response of success tothe processor 161 that executes the configuration change program 212(704).

Next, when the processor 210 receives the configuration informationchange request (705 “Y”), the processor 210 confirms that the I/O is setin the temporary hold state (706). This is because the processor 210does not change the configuration information 221 incorrectly duringinput-output processing. Further, the processor 210 changes theconfiguration information 221 (707). As a result, when the processor 210was able to change the configuration information normally (708 “Y”), theprocessor executes the processing of the step 704.

Moreover, when the processor 210 received the I/O restart request (709“Y”), the processor confirms that the I/O is set in the temporary holdstate (710). As a result, when the I/O is set in the hold state (710“Y”), the processor 210 calls the I/O processing program 213 andreleases the I/O temporary hold state, then restarts input-output (711).When the processor 210 was able to change the configuration informationnormally (712 “Y”), the processor executes the processing of the step704. The aforementioned is the normal path of the configurationmanagement program 211.

On the other hand, the configuration management program goes to anabnormal path if No (“N”) is selected in the steps 703, 706, 708, 710,and 712. In any case, an error and its cause are returned to theprocessor 161 (713).

The processor 210 can change the configuration information 221 inaccordance with the instruction of the configuration change controller16 by executing the configuration management program 211 in this manner.

Next, the processing operation of the I/O processing program 215 in thevirtualization switch 11 is described with reference to FIG. 8.

The I/O processing program 213 is called when an input-output requestwas enqueued to the input queue 241, when the input-output request ofthe in-process queue 252 was completed, when the I/O shifts to atemporary hold state by the processing of the configuration managementprogram 211, and when the I/Os temporary hold state are releaseaccording to the I/O restart request.

When the input-output request enqueued to the input queue 241 (801), theprocessor 210 processes the input-output request when the I/O is not setin the temporary hold state (802 “N”). That is, the processor 210converts the address of the virtual volume 100 to the address of the LU131 with reference to the configuration information 221 and enqueues theinput-output request to the output queue 251 (hereafter referred to asthe fact that the input-output request was processed). Then theprocessor enqueues the output-output request to the in-process queue 252(803). Subsequently, the processor 210 executes an event wait (804) andwaits for the following activation cause.

On the other hand, when the I/O is set in the temporary hold state (802“Y”), the processor 210 enqueues the input-output request to the on-holdqueue 242 (805) and executes the processing of the step 804 and latersteps.

When the processor 210 completed the input-output request of thein-process queue 252 (807 “Y”), the processor dequeues the input-outputrequest from the in-process queue 252 (808). Then the processor confirmswhether the I/O is set in the temporary hold state or not (809). If theI/O is not set in the temporary hold state (809 “N”), the processor 210executes the processing of the step 804 and later steps.

On the other hand, when the I/O is set in the temporary hold state (809“Y”), the processor 210 verifies the in-process queue 252 (810). As aresult of the verification, when the in-process queue 252 was empty (810“Y”), the processor reports the transition completion to an I/O holdstate to the configuration management program 211 (811) and completesthe processing. To the contrary, when the in-process queue 252 is notempty (810 “N”), the processor 210 repeats the processing from the step804.

When the I/O shifts to the temporary hold state (812 “Y”), the processor210 stores that the I/O was set in the temporary hold state (813, 814)and continues the processing from the step 810.

Further, when receiving the restart request (815 “Y”), the processorconfirms whether the I/O is set in the temporary state or not (816). Asa result of the confirmation, if the I/O is set in the hold state, theprocessor releases the I/O temporary hold state and processes theinput-output of the on-hold queue 242, then enqueues to the in-processqueue 252 (817). Then when the in-process queue was enqueued normally(818 “Y”), the processor executes the processing of the step 811 (thatis, completion report). The aforementioned is the processing when theI/O processing program 213 was executed normally.

To the contrary, if the processing of the I/O processing program 213 isnot normal, it indicates that No (“N”) was selected in the steps 814,816, 818. In these cases, because any processing cannot continue, anerror and its cause are returned to the configuration management program211 (819).

In this embodiment, the input-output processing during the change of theconfiguration information 221 that causes data corruption is preventedin the step 610 when the configuration change control program 212 isexecuted and a conflict of the configuration information 221 between thevirtualization switches 11 that causes the data damage is prevented inthe step 603 in the same manner.

Next, a second embodiment is described with reference to FIGS. 9 to 11.

FIG. 9 shows the internal configuration of the virtualization switch 11in a storage system.

FIG. 9 differs from the virtualization switch 11 shown in FIG. 2 in thatthe processor 161 connected to the bus 270 is provided, theconfiguration change control program 212 is stored in the memory 220,and the processor 161 executes this configuration change control program212. In this case, the processor 161 is provided with a timer 165, andthe processing of the configuration change control program 212 isactivated periodically by this timer 165. The timer 165 may be a timerrealized by software.

According to this embodiment, the virtualization switch 11 can functionas the configuration change controller 16 by incorporating theconfiguration change control program 212 and the processor 161 in thevirtualization switch 11 (the virtualization switch 11 that functions inthis manner is also hereafter referred to as the configuration changecontroller 16). When the plural virtualization switches 11 are provided,they are all provided with the aforementioned configuration and canfunction as the configuration change controller 16. If all thevirtualization switches 11 are provided with the configuration changecontrol program 212 and the processor 161 in this manner, theconfiguration change controller 16 in the system shown in FIG. 1 becomesnecessary.

The processor 210 provided in the virtualization switch 11 can use thefunction of the processor 161 simultaneously.

FIG. 10 shows the processing of the configuration change control program212 in the second embodiment. FIG. 10 differs from FIG. 6 in thatarbitration processing 600 is included as the processing operation ofFIG. 10. FIG. 11 shows a detailed flow of the arbitration processing600.

The arbitration processing 600 limits, among the plural processors 161that correspond to the plural virtualization switches 11, processors(hereafter referred to as a processor having a master right) thatexecutes the processing of the step 601 and later steps to oneprocessor. In this embodiment, the configuration change control program212 is also activated by receiving a monitoring packet from anotherconfiguration change control program 16 (incorporated in thevirtualization switch 11) and the timer 165 in addition to aconfiguration change request from the management console 14.

In the flowchart of FIG. 11, first, when the processor 161 received amonitoring packet (1101 “Y”), the processor returns a response packet tothe configuration change controller 16 that sent this monitoring packet(1102), completes the arbitration processing, and also completes theprocessing of the configuration change control program 212 (1110).

The monitoring packet and the response packet include the ID of the sentconfiguration change controller 16 and the ID (hereafter referred to asthe ID of a controller having a master right) of the configurationchange controller 16 that incorporates the processor having the masterright. These identifiers identify the configuration change controller16. For example, the identifiers are IP addresses which thecommunication unit 260 has. When the processor 161 received aconfiguration change request from a management console (1103 “Y”), theprocessor 161 asserts the master right to another configuration changecontroller 16, that is, the processor 161 sends the monitoring packet inwhich the ID of the local configuration change controller 16 wasrecorded as the ID of the controller having the master right to all theconfiguration change controllers 16 and waits for a response (1104). Fora normal end, that is, when the processor did not receive the responsepacket that has the ID other than the local configuration changecontroller 16 as the ID of the controller having the master right (1105“Y”), the processor completes processing and shifts to the processing ofthe step 601 of the configuration change control program 212 (1106).

In the step 1103, when the program was called with the timing of thetimer 165, the processor 161 monitors the remote configuration changecontroller 16, that is, sends the monitoring packet to the controllerhaving the master right (1107). For the normal end, that is, when theprocessor received the response packet that has the ID of the controlleras the ID of the controller having the master right (1108 “Y”), theprocessor advances to the processing of step 1110 and completes theprocessing. The aforementioned is the normal case of the arbitrationprocessing.

To the contrary, for abnormal processing, that is, when negation (“N”)was selected in the steps 1105, 1108, because there is anothercontroller having the master right, the processor sends notice as suchto the management console 14 (1109), the processor goes to theprocessing of the step 1110 and completes the processing.

Because another processing operation is the same as the aforementionedfirst embodiment, the description is omitted.

Thus, according to the second embodiment, the controller having themaster right is limited to only the controller 16 that received aconfiguration change request from the management console 14 at first bythe processing of the steps 1104, 1102. That is, even if the pluralprocessors 161 execute the configuration change control program 212 in asystem, they can change the configuration information 221 withoutgenerating a conflict.

When a fault occurred in the controller 16 having the master right, thesystem administrator can identify this fault via the management console14 through the processing of the steps 1107, 1102.

Next, a third embodiment is described with reference to FIGS. 12 to 18.

FIG. 12 shows the internal configuration of the virtualization switch 11in a storage system.

As compared with the virtualization switch 11 shown in FIG. 9, FIG. 12differs in that a temporary hold control table 223 is provided, theplural in-process queues 252 are provided, and double-bufferedconfiguration information 221 a, 221 b is provided. The processor isprovided with the timer 215.

The temporary hold control table 223 controls whether the input-outputis held temporarily by storing the result of associating each entry ofthe configuration information 221 with the in-process queue 252. Thereason why the plural queues to be processed 252 are provided is tolimit a completion wait and input-output held temporarily by dividingthe in-process queues 252 according to an input-output destinationaddress. The configuration information 221 has double-buffereds toshorten the processing time in the I/O temporary hold state by switchingthe face of the configuration information 221. Further, the timer 215 isinstalled to set the processing of the I/O processing program 213 by theprocessor 210 and detect a timeout of the I/O temporary hold state.

FIG. 13 shows the communication protocol between the configurationchange controller 16 and the virtualization switch 11 in the thirdembodiment.

In this embodiment, because only one virtualization switch enables aconfiguration change control function in the plural virtualizationswitches 11, the communication becomes necessary between thevirtualization switch 11 in which the configuration change controlfunction is effective and the virtualization switch 11 the function ofwhich is not effective. The communication protocol is the example shownin FIG. 13. The same communication is performed even in the samevirtualization switch 11. Accordingly, you are requested to assume thatthe “Configuration Change Control Function” shown in FIG. 13 indicatesthe processor 161 of FIG. 12 or the configuration change control program212 that is executed there and the “Virtualization Switch 11” indicatesthe processor 210 in the local and remote virtualization switches 11 orthe configuration management program 211 that is executed there and torefer to both.

FIG. 13 differs from the communication protocol shown in FIG. 5 in thatsteps 531 to 533 are added between the steps 501 and 502. Further, aconfiguration information switching request is sent instead of sending aconfiguration information change request in the steps 506 to 508 of FIG.5.

In step 530, the processor 161 sends the configuration informationdifference 222 to all the virtualization switches 11 (530). All thevirtualization switches 11 store the configuration informationdifference and create the configuration information 221 (531). Thevirtualization switch 11 creates the face (221 b here) that is not usedby the I/O processing program 213 of the configuration information 221.Subsequently, the virtualization switch 11 returns a configurationinformation difference receiving completion report to the configurationcontroller 16 (532, 533).

Further, in step 507′, the virtualization switch 11 switches theconfiguration information 221 a to the configuration information 221 b.The processing time can be reduced because the processing during an I/Otemporary hold state (between the steps 503 and 511) is completed onlyby switching the face of the configuration information 221 in the step507′ in this way.

FIG. 14 shows an example of the temporary hold control table 223 in FIG.12.

The temporary hold control table 223 has a plurality of entries. Eachentry has a temporary hold state 2231 and a in-process queue ID 2232.The temporary hold state 2231 and the in-process queue ID 2232 are bothprovided to limit a completion wait and input-output held temporarily.As the initial values of the temporary hold state 2231 and thein-process queue ID 2232, the processor 210 stores the ID of thein-process queue 252 that is not in a temporary hold state and empty(not associated with the temporary hold control table 223) respectivelywhen the virtualization switch 11 is initialized or when the virtualvolume 100 is created. When the empty in-process queue 252 is provided,the processor 210 creates the in-process queue 252 anew and stores theID as the ID 2232.

FIG. 15 shows the configuration information 221 in FIG. 12.

FIG. 15 differs from the table with the table shown in FIG. 4 in that anindex 411 of the control table 223 is provided. The index 411 specifiesthe entry of the temporary hold control table 223. The entry of theconfiguration information 221, that is, the address range on the virtualvolume 100 is associated with the temporary hold state 2231 and thein-process queue 252 through the entry of the temporary hold controltable 223. Accordingly, whether input-output is held temporarily peraddress range can be controlled.

FIG. 16 shows the processing of the configuration change control program212 in the virtualization switches 11 shown in FIG. 12.

FIG. 16 differs from FIG. 10 in that steps 625 and 626 are inserted nextto the arbitration processing (step 600) and step 627 is inserted nextto the step 602. Further, steps 603′ and 612′ are included instead ofthe steps 603 and 612.

The processing of the steps 625 and 626 is the same as the procedures530, 533 of FIG. 13. In the step 627, the processor 161 issues an I/Otemporary restart request once (609) and retries the processing of thestep 601 and later steps when the processor 210 detects a timeout of theI/O temporary hold state during processing of the I/O processing program213 (627 “Y”).

In this step 627, an I/O temporary hold state can be set prior to aninput-output timeout by the host processor 12. Further, in the steps603′ and 612′, the processing of switching the configuration informationfrom 221 a to 221 b is requested from the virtualization switch 11instead of rewiring the configuration information 221. Because theswitching processing of the configuration 221 is completed in a shortertime than rewriting the configuration information to a memory, the timeof the I/O temporary hold state can be reduced as a result.

FIG. 17 shows the processing of the configuration program 211 in thevirtualization switch 11 of FIG. 12.

FIG. 17 differs from the flowchart shown in FIG. 7 in that steps 714 to718 were added next to the step 709 “N”. In the step 707, among theconfiguration information 221, the face (221 b) which the I/O processingprogram 213 does not use is updated. The processing of steps 714 “Y” to704 is the same as the steps 507′, 508′ of FIG. 13.

The configuration management program 211 is called when a request fromthe I/O processing program 213 is issued in addition to an I/O temporaryhold request, an I/O restart request, a configuration information updaterequest, and a configuration information switching request. When thisrequest is provided, processing goes to the step 601 “N”. Subsequently,the processor 210 transfers the request of the I/O processing program213 to the configuration change controller 16 (718). This processing canretry the processing in the configuration change control program 212 ona timeout of an I/O temporarily stopped state.

Further, the processing of the steps 705 to 704 is the same as theprocedures 531, 532 of FIG. 13.

FIG. 18 shows the processing of the I/O processing program 213 in FIG.12.

FIG. 18 differs from the flowchart of FIG. 8 in the processing contentsof the steps 802, 803, 805, 808, 809, 810, 813, 816, 817.

In steps 802′, 809′, 816′, the processor 210 determines that the I/O isset in a temporary hold state by checking the temporary hold state 2231of the entry the index 411 in the configuration information 221specifies. This can control whether the input-output is held temporarilyor not per address range of the virtual volume 100 that is aninput-output destination.

Further, in steps 803′ and 808′, the processor 210 uses the in-processqueue 252 specified with the ID 2232 of the in-process queue of theentry which the index 411 in the configuration information 221 specifiesas the in-process queue 252.

Further, in step 813′, the processor 210 changes a state so that onlythe input-output for an area on the virtual volume 100 defined accordingto the contents of the configuration information difference 222 can beset in the temporary hold state. Specifically, the processor 210 listsup the index 411 of the configuration information 221 a that correspondsto all entries registered in the configuration information difference222 and changes all the temporary hold states 2231 of the entry thatcorresponds to these indexes 411 during a temporary hold. As describedalready, in the step 802′, this temporary hold state 2231 is checked.Accordingly, the output-output request that corresponds to thistemporary hold state 2231, that is, only the input-output requestaffected by the configuration information difference 222 is held.

In step 810′, the processor 210 lists up the index 411 of theconfiguration information 221 a that corresponds to all entriesregistered in the configuration information difference 222 and checksall the in-process queues 252 specified by the in-process queue ID 2232of the entry that corresponds to these indexes 411, then determineswhether the in-process queue 252 becomes empty or not.

In step 805′, the processor 210 enqueues to the in-process queue 252shown in the in-process queue ID 2232 and sets the timer 215. In step817′, the processor 210 resets the timer 215, resets the temporary holdstate 2231 held in step 813′, processes the input-output enqueued to theon-hold queue 242, and enqueues to the queue be processed 252 shown inthe in-process queue ID 2232.

In the step 820 “Y”, the processor 210 starts the processing of the I/Oprocessing program 213 on the timeout of the timer 215. In this case,the processor 210 executes the processing of the step 819 and returns anerror to a configuration management program. The processor 210 executesthe processing of the step 818.

Because other aspects are the same as the second embodiment, thedescription is omitted.

In the third embodiment, the time of the I/O temporary hold state can bereduced in the steps 625 and 603′ in this manner. The time of the I/Otemporary hold state can be limited by releasing the I/O temporary holdstate and retrying the processing in the steps 805′, 820, 627. This canprevent the input-output timeout or performance deterioration in thehost processor 12.

Further, in this embodiment, it is possible to limit input-outputtemporarily held in steps 813′ and 802′ to a range affected by theconfiguration change. Accordingly, it is possible to preventdeterioration of performance by the configuration change.

As a modification example of this embodiment, there is also a method forbeing not provided with the aforementioned double-buffered configurationinformation 221 and not executing the processing of the step 625. Forexample, the processing of the step 603 (rewriting of the configurationinformation 221 in the first embodiment) may also be performed insteadof step 603′ in which the configuration information 221 is switched. Inthat case, although the time of the I/O temporary hold state isprolonged, the capacity of the memory 220 can be suppressed.

Next, a fourth embodiment is described with reference to FIGS. 19 to 26.

FIG. 19 shows the internal configuration of the virtualization switch 11in the fourth embodiment.

FIG. 19 differs from the configuration shown in FIG. 12 in that a copyprocessing program 214 and a copy progress table 224 are added and theone-face configuration information 221 is provided.

When the copy processing program 214 is executed by the processor 210and changes the LU 1311 to which the virtual volume 100 corresponds tothe other LU 1312, the program copies data from the LU 1311 to the LU1312. The copy progress table 224 is used to manage advancement of thecopy processing in the copy processing program 214 and has a pluralityof entries. Each entry corresponds to the real area 132 that constructsthe LU 131. In this embodiment, the configuration information difference222 is generated during I/O temporary stop. Accordingly, theconfiguration information 221 has only one face. Other aspects are thesame as FIG. 12.

FIG. 20 shows an example of the configuration information 221 in thevirtualization switch 11 of FIG. 19.

As compared with FIG. 15, in FIG. 20, a total of three pairs of the LUaddress and offset consisting of one pair of 44 and 45 for the Readcommand and two pairs of 46 and 47, and 48 and 49 for the Write commandare provided. In this embodiment, the correspondence of the virtualvolume 100 differs in the Write and Read commands due to the progress ofthe copy processing program 214. Further, in the case of Writeoperation, because dual writing is also performed to the LU 1311 and theLU 1312, two pairs of the address and offset for the Write command areprepared.

FIG. 21 shows the processing of the configuration change control program212 in the virtualization switch 11 of FIG. 19.

As compared with the flowchart of FIG. 16, in FIG. 21, steps 631 to 633and 635 are added between the steps 627 and 603 and step 634 is addedbetween the steps 606 and 607. Further, the processing of the steps 603,612 is performed instead of the steps 603′, 612′.

In the copy processing 631, the processor 161 starts the copy processingprogram 214. When copying is performed without any error (632 “Y”), theprocessor 210 refers to the copy progress table 224 and creates theconfiguration information difference 222. The details of differencecreation will be described later with reference to FIG. 26.

In the step 634, the processor 161 determines whether the copying fromthe LU 1311 to the LU 1212 was all completed by checking the copyprogress table 224. When the copying is completed (634 “Y”), theprocessor 161 executes the processing of the step 607 and completes theprocessing. On the other hand, when the copying is not completed (634“N”), the processor 161 repeats the processing from the step 601.

When the copying ended abnormally (632 “N”), the processor 161 performsrecovery processing (635). Specifically, the processor 161 creates thedifference 222 so that the configuration information 221 beforeexecuting the configuration change control program 212 can be returnedand issues a configuration change request to all the virtualizationswitches 11, then waits for the completion. Further, the processor 161executes the processing of the step 614 and later steps and reports anerror to a management console, then completes the processing.

FIG. 22 shows the detailed processing operation of the copy processing631 shown in the flowchart of FIG. 21.

First, the processor 161 sends a copy start request to the processor 210and activates the copy processing program 214 (6312) after activatingthe configuration change control program 212 and then activating firstcopy processing 6311 (6311 “Y”). with the copy start request, theprocessor 161 instructs the address range in which copying is performedby the activation of the copy processing program 214 to the processor210.

On the other hand, when no initial activation is performed (6311 “N”),the processor sends the copy restart request to the processor 210 andactivates the copy processing program 214 (6313). Also in this case, theprocessor instructs the address range in which the copying is performedby the activation of this copy processing program 214 to the processor210. In both cases, the processor 161 subsequently enters an event waitstate (6314). The events for which the processor 161 waits are thecompletion of the copy processing program 214 and the receiving of acopy priority request or a copy interrupt request in step 718 to bedescribed later. When the processing of the copy processing program 214was completed (6315 “N”, 6316 “N”), the processor 161 shifts to step632.

For the copy interrupt request (6315 “Y”), the processor sends the copyinterrupt request to the processor 210 and activates the copy processingprogram 214 (6317). For the copy priority request, the processorinterrupts the copying once and performs the processing from the step6315. Subsequently, because the copy processing must be reactivated, theprocessor 161 executes the step 6316 “Y” and repeats the processing ofthe step 6313 and later steps. At that time, because the processortransfers an address to be copied preferentially from the processor 210,it specifies the range that includes the address in the step 6313.

As described in the steps 6312 and 6313, the copy processing program 214is activated plural times. Further, in the step 6317, the copying isinterrupted once. Consequently, the time the input-output is heldtemporarily can be reduced.

FIG. 23 shows the processing of the configuration management program 211in the virtualization switch 11 of FIG. 19.

In this embodiment, because the face of the configuration information221 is not switched during I/O temporary hold state, the processing ofthe configuration management program 211 is similar to that shown in thefirst embodiment. Accordingly, when FIG. 23 is compared with theflowchart of FIG. 7, FIG. 23 differs from FIG. 7 in that step 718 isadded.

In this embodiment, the timing the configuration management program 211is called is caused by a copy priority request or a copy interruptrequest from the I/O processing program 213 in addition to the case ofFIG. 7. In either case, the processor 210 determines as No in step 709and issues the copy priority request or copy interrupt request to theprocessor 161 (718). For the copy priority request, however, because theprocessor 210 transfers an address to be copied preferentially from theI/O processing program 213, it also transfers the address to theprocessor 161.

Thus, the configuration management program 211 reflects a request causedby the processing of the I/O processing program 213 in the processing ofthe configuration change control program 212 by the processing of thestep 718.

FIG. 24 shows the processing of the I/O processing program 213 in thevirtualization switch 11 of FIG. 19.

In comparison with the flowchart shown in FIG. 18, in FIG. 24, step 806is added next to the step 805′ and step 821 is added next to the step820.

In the step 806, the processor 210 calls the configuration managementprogram 211 and requests copy priority when the accepted input-output isWrite operation. In that case, the processor transfers the address rangewritten by the accepted input-output to the configuration managementprogram 211.

In this embodiment, because the timeout (820 “Y”) of the timer 215 iscaused by the copy processing, in the step 821, the processor 210 callsthe configuration management program 211 and requests a copy interrupt.

When the copy processing by the input-output held temporarily isinterrupted or made to take preference by the processing of the steps806 and 821, the time the input-output is held temporarily can bereduced.

In step 806, by making a copy interrupt request when the acceptedinput-output is read, the time the input-output is held temporarily canbe further reduced (In this case, the step 821 becomes unnecessary). Thecopy processing, however, is as delayed as this operation.

FIG. 25 shows the processing of the copy processing program 214 in thevirtualization 11 of FIG. 19.

The copy processing program 214 is activated at the timing of thecompletion of a copy start request, a copy restart request, and a copyinterrupt request from the configuration change control program 212, andthe input-output that copies data actually. When the copy start requestis provided (901 “Y”), the processor 210 initializes all the entries ofthe copy progress table 224 to a value (“0” in this example) thatindicates Uncopied (902). Subsequently, the entry of the copy progresstable that corresponds to the address range transferred from theconfiguration change control program 212 is set to a value (“1” in thisexample) that indicates Being Copied (903).

Further, the processor 210 issues an input-output instruction (hereafterreferred to as COPY I/O) that performs copying to the area set to “1”(Being Copied) in the copy progress table 224 (904). Specifically, theinput-output request that performs copying is generated and enqueued tothe output queue 251. Further, the processor 210 waits for an event(905). Among the activation timings of the copy processing program 214,the processor waits for a request other than the copy start request andexecutes the processing of step 906 and later steps. Further, when thecopy restart request was provided (906 “Y”), the processor 210 repeatsthe processing of the step 903 and later steps.

When COPY I/O is completed (907 “Y”), the processor 210 confirms thatthe result of this input-output ends normally (908 “Y”) and updates thecopy progress table 224 (909). Specifically, the processor sets a value(“4” in this example) that indicates Copied in an entry where COPY I/Owas completed and sets a value that indicates a copy interrupt (in thisexample, “0” (Uncopied) in the entry in which “2” is stored). Further,when there is no entry in which “1” (Being Copied) in the entry of thecopy progress table 224, the processor 210 returns a response of successto the configuration change control program 212 and completes theprocessing (911). When there is an entry being copied (910 “N”), theprocessor 210 repeats the processing from the step 904.

When the copy interrupt request is received (912 “Y”), the processor 210sets “2” (Copy Interrupted) in all entries where “1” of the copyprogress table 224 is set and repeats the processing from the step 905.

When COPY I/O ended abnormally (908 “N”), the processor 210 sets “0”(Uncopied) in all entries where “1” (Being Copied) and “2” (CopyInterrupted) of the copy progress table 224 are set (914). Then an errorand its cause are returned to the configuration change control program212 and the processing terminates (915).

The copy processing program 214 can copy data by the processing of theCOPY I/O of the step 904 in this manner.

The operation principle of data migration according to the fourthembodiment is described with reference to FIG. 26.

FIG. 26A shows the interrelationship between the virtual volume 100 andthe real area 132, and the interrelationship with the copy progresstable 224 in the copy processing 631.

The arrow from the real area 132 to the virtual volume 100 shows theinterrelationship in a Read request to the virtual volume 100 and thearrow from the virtual volume 100 to the real area 132 shows theinterrelationship in a Write request to the virtual volume 100. An arrowmarked by a dotted line shows that the input-output to this arrow isheld.

Arrows appear from an area 1001 on the virtual volume to a real area13211 and a real area 13221. This indicates that the same data iswritten dually to the two real areas 13211, 13221 when the Write requestis issued to 1001.

Further, the arrows from real areas 13212, 13213 to real areas 13222,13223 indicate copying is performed in this direction. 2241 a, 2242 a,2243 a, 2244 a of the copy progress table correspond to areas 1001,1002,1003, 1004 on the virtual volume.

FIG. 26A shows that 2241 a is set to Copied “4” and the copying from thereal area 13211 to real 13221 is completed regarding an area 100-1.Regarding the copied area, for Read, data is written from a shiftdestination, that is, 1311, and, for Write, data is written to bothshift source and shift destination, that is, both 1311 and 1312. This isbecause the latest data is left in the LU 1311 even when the shift intothe LU 1312 failed halfway.

Further, 2242 a to 2244 a are set to Being Copied “1” and indicate thatdata is copied currently from these corresponding real areas 13212 to13214 to the real areas 13222 to 13224. 2245 a is set to Uncopied “0”and indicates copying is not performed from a real area 13215 to a realarea 13225.

FIG. 26B shows the copy progress table 224 after the host processor 12issued a Write request to the area 1004. When Write to the area beingcopied is received, the processor 210 requests copy priority through thestep 806 of the I/O processing program 213 (FIG. 24). Subsequently, theprocessor 161 first performs copy interrupt processing 6317 of the copyprocessing 631. As a result, the processor 210 executes the step 912 ofthe copy processing program 214. Because the mark “2” of CopyInterrupted is set in the area of Being Copied “1” of the copy progresstable 224, the state of Table 224 b occurs.

Subsequently, the processor 210 waits for the completion of COPY IO andexecutes the processing of the step 909, then updates the copy progresstable 224. In this case, if copying to the real area 13222 only iscompleted, the entry 2242 b that corresponds to this is updated fromCopy Interrupted “2” to Copied “4” and another entry is updated from themark “2” of Copy Interrupted to the mark “0” of Uncopied. Further, theprocessor 161 performs the processing of the step 6313 in the copyprocessing 631 and specifies the address of the real area 13214 to thearea 1004. Because the processor 210 performs the processing of the step906 and later steps of the copy processing program 213, the processor210 sets the mark “1” of Being Copied in 2244 b that corresponds to thearea 1004. Thus the processor enters the state of FIG. 26C. That is, theprocessor 210 preferentially copies data from the real areas 13214 to13224.

Because other processing is the same as the third embodiment, thedescription is omitted.

As described above, the allocation destination of the virtual volume 100can shift the allocation destination of the virtual volume 100 from theone LU 1311 to the other LU 1312 during system operation.

Next, a fifth embodiment is described with reference to FIGS. 27 and 28.

FIG. 27 shows the overall configuration of a storage system.

As compared with the system shown in FIG. 1, in FIG. 27, the independentconfiguration change controller 16 is removed, and a configurationchange control program 16′ is provided in each virtualization switch 11.Further, a storage device 135 has a copy control unit 136.

The storage device 135 differs from the storage device 13 and has thecommunication unit 260. The storage device 135 is connected to the LAN15 and has the copy control unit 136. This enables data to be copiedfrom an LU 131 a within the local storage device 135 to an LU 131 c.Desirably, the copy control unit 136 provided in the storage device 135should perform copy processing while permitting the input-output fromthe host processor 12 to the copy source UL 131 (hereafter referred toas ‘Copiable’ during online operation). Needless to say, ‘Copiable’ neednot be required during online operation. When copying is not enabledduring this online operation, the time the input-output is heldtemporarily is prolonged.

The copy control unit 136 receives the designation of performing thecopy processing from which LU 131 to which LU 131, for example, ofcopying data from the LU 131 a to the LU 131 b as well as the request ofthe start or completion of the copy processing from the configurationchange controller 16 or the management console 14 via the communicationunit 260.

FIG. 28 is a flowchart showing the processing of the configurationcontrol program 212 in the fifth embodiment.

FIG. 28 differs FIG. 16 in that the processing of the step 640 isinserted after the step 627.

In the processing of the step 640, the processor 161 requests to thecopy control unit 136 the completion of the copy processing from the LU131 a to the LU 131 b and waits for the completion report. When copyingis enabled during online processing, the system administrator canrequest copy start at an optional period before the step 640. Whencopying is disabled during online processing, the processor 161 requeststo the copy control unit 136 the start of the copy processing in thestep 640 prior to the request of the completion. By hastening therequest of the copy start, it can be anticipated that a wait completionreport in the step 640 is returned quickly.

The processors 161 and 210 perform the same processing as the thirdembodiment and changes the configuration so that the virtual volume 100can correspond to the LU 131 a and the LU 131 b. Because otherprocessing is the same as the third embodiment, the description isomitted.

According to the fifth embodiment, by utilizing the copy function whichthe storage device 135 has, the allocation destination of the virtualvolume 100 can be shifted from the LU 131 a to the LU 131 b even if thefunction of copying the LU 131 to the virtualization switch 11 is notprovided.

Next, yet another example (sixth embodiment) of a storage system isdescribed with reference to FIG. 29.

In this example, a storage device 137 has a virtualization function.Each storage device 137 has the storage device 13, copy control unit136, and communication unit 260 and the virtualization switch 11 shownin FIG. 27. Each virtualization switch 11 incorporates the configurationchange control program 16′ similarly to the aforementioned FIG. 27 andimplements the configuration change control function. Because otheraspects are the same as the fifth embodiment, the description isomitted.

According to this example, while the system is operating by avirtualization switch having a redundant configuration, a storage devicethat can shift the allocation destination of the virtual volume 100 fromthe LU 131 a to the LU 131 b can be realized.

Although several embodiments have been described above, the presentinvention can be modified variously and executed without being limitedto the above examples. For instance, in the examples of FIGS. 12 and 19,one of the processors 161 and 210 is omitted and the remaining otherprocessor can be used for the processing of a program at the same time.

1. An allocation method for allocating a plurality of logical volumes toform a plurality of virtual volumes among a plurality of virtualizationswitch apparatuses, the virtualization switch apparatuses being coupledto a plurality of host computers and to a storage device, the logicalvolumes related to at least one disk drive in the storage device, thevirtual volumes processing input-output requests from the hostcomputers, the method comprising steps of: issuing, to the plurality ofvirtualization switch apparatuses, a request for completing allinput-output requests received from the host processors that are beingprocessed by the virtualization switch apparatuses and temporarilyholding any subsequent input-output requests received from the hostprocessors; receiving, from the plurality of virtualization switchapparatuses, a completion report of the input-output requests beingprocessed by the virtualization switch apparatuses in response to therequest for completing; sending an instruction of an allocation changeof logical volumes to all the virtualization switch apparatuses uponreceiving completion reports from all the virtualization switchapparatuses to which the request for completing was issued; receiving acompletion report of the allocation change from all the virtualizationswitch apparatuses; and sending an instruction to all the virtualizationswitch apparatuses for releasing the input-output request that are beingtemporarily held.
 2. An allocation method according to claim 1, whereina table storing configuration information that associates the virtualvolume with the logical volumes that becomes a real area of the storagedevice is prepared in a memory in advance, and when the instruction ofan allocation change is sent, difference information of theconfiguration information is sent, and the virtualization switchapparatus changes the configuration information on a relevant entry ofthe table.
 3. An allocation method according to claim 1, wherein avirtualization switch apparatus that did not send its completion reportis removed from a control range and the allocation of its logicalvolumes is not changed.
 4. An allocation method according to claim 1,wherein whether a subsequently received input-output request is heldtemporarily or not is controlled aiming at an address range including alocation where the allocation is changed on the virtual volume.
 5. Anallocation method according to claim 1, further comprising, for avirtual volume with newly allocated logical volumes, copying data fromlogical volumes previously allocated to the virtual volume to the newlyallocated logical volumes.
 6. A storage system, comprising: a storagedevice that can specify a plurality of logical volumes; a plurality ofvirtualization switch apparatuses that allocate the logical volumes toform a plurality of virtual volumes, and to process input-outputrequests sent from a plurality of host processors to one of the virtualvolumes; and a configuration change controller for changing anallocation configuration of the logical volumes to the virtual volumes,wherein the configuration change controller includes means forrequesting temporary hold of input-output requests to the virtualizationswitch apparatuses, wherein the virtualization switch apparatus thatreceived the request includes means for completing all input-outputrequests received from the host processors that are being processed bythe virtualization switch apparatus, shifting to a state of temporarilyholding subsequently received input-output request from the hostprocessors, and returning a completion report of processing of theinput-output requests to the configuration change controller, whereinthe configuration change controller includes means for instructing anallocation change of the logical volumes to the virtual volume to thevirtualization switch apparatus when receiving the completion reportfrom all the virtualization switch apparatuses to which a request wasissued.
 7. A storage system according to claim 6, wherein theconfiguration change controller includes: a configuration change controlprogram that includes the request means, means for receiving thecompletion report from the virtualization switch apparatus, and thechange instruction means; a processor that executes the configurationchange control program; and a memory that stores a configurationinformation table registering configuration information that associatesthe virtual volume with the logical volumes that becomes a real area ofthe storage device and a difference information table recording adifference before and after the change of the configuration information,the virtualization switch apparatus, includes: a configurationmanagement program that performs processing of a configuration change; aprocessor that executes the configuration management program; and amemory that stores a configuration information table registering theconfiguration information that associates the virtual volume with thelogical volumes that becomes the real area of the storage device and adifference information table recording the difference before and afterthe change of the configuration information, and the configurationchange controller sends the difference information of the configurationinformation to the virtualization switch apparatus with reference to thedifference information table when sending the instruction of theallocation change of the logical volumes, and the virtualization switchapparatus executes the configuration management program by the processorand changes the configuration information of a relevant entry of its ownthe configuration information table in accordance with the receiveddifference information.
 8. A storage system according to claim 6,further including a management console comprised of an input unit thatinputs a request of the change of the configuration information to theconfiguration change controller and a display unit that displays astatus of the configuration change.
 9. A virtualization switch apparatusthat allocates logical volumes of a storage device, forms a plurality ofvirtual volumes from the logical volumes, and processes input-outputrequest sent from a plurality of host processors to one of the virtualvolumes, comprising: a configuration change control program for changinga configuration of associating the virtual volume with the logicalvolume that becomes a real area of the storage device; and a firstprocessor that executes the configuration change control program,wherein the program includes: means for requesting an input-outputrequest temporary hold to a first virtualization switch apparatus beforechanging the configuration of associating the virtual volume with thelogical volumes that becomes the real area of the storage device; meansfor allowing the first virtualization switch apparatus that received therequest to complete all input-output requests received from that hostprocessors that are being processed, shifting to a state of temporarilyholding subsequently received input-output requests from the hostprocessors, and returning a completion report; means for instructing, tothe first virtualization switch apparatus, an allocation change of thelogical volumes to the virtual volume when receiving the completionreport from the first virtualization switch apparatus; means forreceiving the completion report of the allocation change from the firstvirtualization switch apparatus; and means for sending an instruction tothe first virtualization switch apparatus for releasing the input-outputrequest that are being temporarily held.
 10. A virtualization switchapparatus according to claim 9, further comprising: a memory storing aconfiguration information table registering configuration informationthat associates the virtual volume with the logical volumes that becomesthe real area of the storage device and a difference information tablethat records a difference before and after a change of the configurationinformation; a configuration management program for receiving a requestfrom the configuration change control program to temporarily holdchanging input-output requests and change configuration information; anda second processor that executes the configuration management program,wherein contents of the configuration information table are updated byexecuting the configuration management program by the second processor.11. A virtualization switch apparatus according to claim 10, wherein thefirst processor and the second processor are comprised of the sameprocessor.
 12. A virtualization switch apparatus according to claim 9,wherein the configuration change control program further comprises meansfor performing arbitration processing to limit the first processor thatexecutes the respective means of the configuration change controlprogram.
 13. A virtualization switch apparatus according to claim 10,wherein the configuration information table comprised of a plurality offaces is prepared and a table of each face is switched.
 14. Avirtualization switch apparatus according to claims 9, furthercomprising: when changing a configuration from the logical volumes towhich the virtual volume corresponds to other logical volumes, a copyprocessing program for copying and processing data to the other logicalvolumes; and a copy progress table that manages a progress status of thecopy processing of the data using the copy processing program.
 15. Astorage device comprising a plurality of logical volumes for providing areal storage area and a virtualization switch apparatus that allocatesthe logical volumes, forms a plurality of virtual volumes, and processesinput-output requests from a plurality of host processors to one of thevirtual volumes, wherein the virtualization switch apparatus includes:means for requesting an input-output temporary hold to a firstvirtualization switch apparatus before changing a configuration ofassociating the virtual volume with the logical volumes that becomes areal area of the storage device; means for allowing the firstvirtualization switch apparatus that received the request to completeall input-output requests received from the host processors that arebeing processed, shifting to a state of temporarily holding subsequentlyreceived input-output requests from the host processors, and returning acompletion report; means for instructing an allocation change of thelogical volumes in regard to the virtual volume to the firstvirtualization switch apparatus when receiving the completion reportfrom the first virtualization switch apparatus; means for receiving thecompletion report of the allocation change from the first virtualizationswitch apparatus; and means for sending an instruction to the firstvirtualization switch apparatus for releasing the input-output requestthat are being temporarily held.
 16. A storage device according to claim15, wherein there are provided a configuration change control programfor realizing each of the above means and a processor that executes theprogram.
 17. A storage device according to claim 15, wherein there isprovided a copy control unit for copying data from logical volumesoriginally allocated to a virtual volume to other logical volumes thatare subsequently allocated to the virtual volume.
 18. A change methodfor allocation of a plurality of logical volumes of a storage device toa virtual volume in a plurality of virtualization switch apparatusesthat process input-output from a plurality of host processors to thevirtual volume, comprising the steps of: issuing, to the plurality ofvirtualization switch apparatuses, a request for temporarily holdinginput-output requests received from the host processors after a certainpoint of time; making the respective virtualization switch apparatuseschange the allocation of the logical volumes on the condition that areport indicating completion of the processing of all input-outputrequests is received from the respective virtualization switchapparatuses; and releasing input-output requests that are beingtemporarily held after the completion report of the allocation change isreceived from the respective virtualization switch apparatuses.
 19. Achange method according to claim 18, wherein the step of inputting aninstruction of a configuration change from a management console isincluded and the request for temporarily holding the input-output isissued in accordance with the input instruction.
 20. A program for aconfiguration change that changes allocation of a plurality of logicalvolumes of a storage device to a virtual volume in a storage systemincluding a plurality of virtualization switch apparatuses that allocatethe logical volumes, form a plurality of virtual volumes, and processinput-output from a host processor to one of the virtual volumes,comprising: means for issuing, to the plurality of virtualization switchapparatuses, a request for completing all input-output requests receivedfrom the host processors that are being processed by the virtualizationswitch apparatuses and temporarily holding any subsequently receivedinput-output requests received from the host processors; means forreceiving, from the plurality of virtualization switch apparatuses, areport indicating completion of the processing of the input-outputrequest in response to the request for completing; means for instructingthe allocation change of the logical volumes of the storage device toall the virtualization switch apparatuses when receiving the completionreport from all the virtualization switch apparatuses to which therequest was issued; means for receiving the completion report of theallocation change from all the virtualization switch apparatuses; andmeans for sending an instruction to all the virtualization switchapparatuses for releasing the input-output request that are beingtemporarily held.